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ABSTRACT 

A study vasiade to deteraine whether or not educable 
mentally retarded students at the primary and intermediate levels 
attend to and gain knowledge from filmstrips* or ifr in fact r their 
test performance variance is distributed randomly. The subjects are 
individually shown a tape-synchronized filmstrip; they are 
adminisfterdd both a pretest and a posttest (answers confirmed oh 
posttest only) . Overall, the results confirm the research assumption 
that viewing a filmstrip is related to the subject's performance as 
measured by criterion referenced itens administered in a 
pretest/posttest design. However, a significant difference in 
performance i$ found between primary and intermediate groups, it Is 
suggested that there is an age level or maturity level below which 
cognitive gain. traceable to a mediated presentation may not occur. 
(Author/HCM) 
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THE EFFECTS OF PRETESTING EDUCABLE MENTALLY HAKDICAPPED CHILDREN 

IN PILMSTRIP EVALUATION 



ABSTRACT 



The purpose of this study was to determine whether or not educatle 
mentally retarded students at the primary and intermediate levels at- 
tend to and gain knowledge from filmstrips, or if, in fact» their test 
performance variance is distributed randomly, The subjects were in- 
dividually shown a tape-synchronized filmstrip; they were administered 
both a pretest and a posttest (answers confirmed on posttest only) 1-^^-^^^^^^^-^^^ - 

Overall, the results confirm the research assumption that viewing 
a filmstrip is related to the subject's performance as measxired by 
criterion referenced items administered in a pretest/posttest design • 

However, a significant difference in performance was found between 
primary and intermediate groups. It was suggested that there is au age 
level or maturity level below which cognitivfi gain traceable to a 
mediated presentation may not occur. 
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SPEC lAL REPORT No. 736 
COMPUTER-BASED PROJECT for the EVALUATION 
of MEDIA for the HANDICAPPED 

T i t I 6 ! THE EFFECTS OF PRETESTING EDUCABLE MENTALLY HANDICAPPED CHILDREN 

IN FILMSTRIP EVALUATION 

BACKGROUN D 

The Computer Based Project for the Evaluation of Media for the Hai^dicappodi 
based on contract ffOEC-9-423617-4357 (616) between the Syracuse (N,y/) City School 
District and the Media Services and Captioned Films Branch, Bureau of Education 
for the Handicapped (United States Office of Education) for the five year period 
July 1, 1969 through June 30/ 1974, The major goal is to improve the instruction 
of handicapped children through the development and use of an evaluation system 
to measure the instructional effectiveness of films and other materials with 
educable mentally handicapped (EMH) children, in-service training and media support 
for special teachers, and studies related to the evaluation process and the 
populations used. 

The Project has concentrated on the. 600 films and 200 film;^ trips from the 
Media services and Captioned Films (BEI/ - USOE) depository; however , specific 
packages from Project LIFE, various elementary math curricula, and selected 
programs from Children's TV Workshop have also been evaluated! The evaluation 
model ueed requires that; 1) objectives of materials be specified and written; 

2) instruments be constructed to test and measure effectiveness; and; 3) children 
be the major sources of evaluation information. A number of instruments and 
methodologies are employed in the gathering of cognitive and affective data from 
900 EMH children and 80 special teachers to make the effectiveness deoisione. 
Over half of the EMH population can neither read or write; therefore, a unique 
Student Response System (SRS) ia employed, consisting of a twenty station 6. E.- 
1000 SRS which can be operated in a group or individual recording mode and is 
connected to a remote computer system. The computer capabilities consist of 
remote telephone connections to the Rome (N.Y.) Air Development Command, the 
Honeywell time-shared network, and the Schenectady (N,Y.) G E Research and 
Development Center; and batch mode capabilities of the Syracuse City Schools, 
Syracuse University, and various ccrronercial sources. 

In-service and media support activities provide on-the-job training for 
teachers, teacher aides, equipment, and materials to the special teachers in 
the city schools. The research activities have centered around investigations 
and special problems related to the development of the evaluation model. The 
four major areas considered ares 1) testing effects, 2) captioning effects, 

3) special student characteristics; and, 4) evaluation procedures validation. 

Documentation of the major activities appear in the five annual reports 
and the 600 evaluations prepared on materials used. Staff members v;Gre encouraged 
to prepare special reports and the attached paper is one of these. The opinions : 
expressed in this publication do not necessarily reflect the position or policy ^^^^^ ^^^^^^^^S' 
of the Computer Based Project, the United States Office of Education, or the 
Syracuse City School District, and no official endorsement by any of the agencies 
should be inferred. 
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IN FlUiSTRIP EVVvLiyATIOH 

The CBP Evaluation ^lodel for Uedlated Uaterlala (Bondi 1972) uses staff 
developed criterion referenced tests as primary evaluation instrutnenta. Based 
on the general assumptions concerning criterion referenced t03t items as ueisuring 
progress^ it is implicit that the tests actually measure students achievement 
of mastery of the content of the media. One needs also to assume that the 
subject will attend to the filmstrlp and the content mastery measurement is 
a result of attention to the film. This study will examine the question: Do 
the EiiR Students at the Primary and Interuadiate levels actually attend to 
and gain from the f ilms trips shovm or is test pertomenco variance distributed 
randomly? If pretest to posttest gain is significaat, attention is assumed* 

The subject population from vlj^ch CBP drav;s its evaluutlon eat*ipl3 is 
educable mentally retarded children served by programs in the Syracuse City 
School District . These children are legally defined as having a measurable 
IQ of 50 to 75 as measured by either the ttISC or Stanford Binet. The subject 
age for the sample used in evaluation model for filmstrips are Primary (CA 6-^8 

years) and Int<^rmedlate (CA ^.^XS yo^rs). 
Problem Statement 

The problem is essentially one of validating our assumptions about the 
effect of the treatmenti i*e*> viewing a filmstrlp - related to the pupilis 

performance as measured by criterion referenced items administered as pretest 
and posttest. The question is whether student achievement as measured by a 
^ .or :A , Ji||as|r 1 ; 



Oj^ Hill the subjects shovra significant difference in performance on 

the second pretest as compared to the first pretest; i.e. 0^ * 
©2 Will the subjects show a significant difference in performance on 

the posttest as compared to the second pretest; i*o. O2 ■ O3/ 
Oy Will there be a difference In the gain between the first pretest and 

second pretest as compared to the gain between the second pretest and 

the posttest; i*e» O2 - Oj^ ■ O3 - Oj^. 
0^, Will there be a difference in the gain between the first pretest and 

posttest as compared to the second pretest and posttest; I.e. 

O3 - « 03 - Oj. 
The null research hypotheses that there will be no difference betvreen 
the variables formulated and stated In the above questions. 

Ten Primary and ten Intericedlato students were dvawn fron the available 
Primary and Intermediate level BUR children at Prescott School, who are normally 
scheduled for fllmstrip evaluation (i.e. I M *» 20). 

The treatment consisted o* Individual showings to the subjects of a tape'- 
syncronir.ed commercially produced filtnstrip, "The Elephant,** via rear screen 
projection using the G.E* AVR 10 as the audio and student response unit and a 
carosell projector to project the slides onto a rear screen mounted in a student 
carrell. 

Ihe G.E. AVRiO is designed to allow data collection by requiring the 
subject to respond by pressing one of five buttons on the unit corresponding 

to numbered items on test queotioas projected as part of the filrastrlp presents-! 
tion* "^^^ device gives conflrtiing green light for a correct response ^f^^^ 

In : 



Tlie Uata wad collected by E. observing the button which the subject ptcoscd 
and recording the response on a score sheet. In e«ich case, only the first 
response to a test it^m vaa recorded. This procedure uas adopted to avoid the 
variability In responding by a subject randomly pleasing response buttons and 
the Inherent problem of machine malfunction with the AVK 10 and the SilS 1,000 
data system to uhlch It Is connected • 

Tlie design Is basically a non-randomized control group deslgv.. 
Primary Oj^ Oj • 20 

Intermediate Oj^ O2 0^ N « 20 

The subjects u^lll be used as their own control, Tficrcforei no assumptions need 
be made about the distribution of the source of variability (Wineri 1962). 
The effect of this procedure is to double the s<i?aple size; l.e«, 20 experimental 
and 20 control subjecta. 

The flriit pretest Oj^ x/as administered follot/ed immediately by the second 
pretest O2 . the fllmtrip t;as then shown and the posttest O3 th-jn adtninls- 
tered. The IteiOs in 02» and O3 consisted of nine Identical criterion 
referehced items written for this filmstrip as a part of the normal CBP BvaVuatiou 
procedure. 

■ yiNDiKc s 

Logically, if there sas no difference between the fltj5t ana second 
observations > but there Is belweea the second and third cbcervatlons , ue can 
conclude that the treatment 5 i.e., the filuistrip ''The Sl?.phant had an effect, 
flie t test was chosen over more c.^r^^^lex testp to asav.re that the first order 
affects v^ere noc "washed out" in the analysis (Xerliu;>er, 196^' Isaac and 
^^•ljic|>^glj^.^ 
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is a function of attending to the inedia, ratheir than ^a,-itiTTctloii of the pretest, 

H03 O2 • " gain 

J3 ')2 « gain 
Ho^ O3 - 0]^ « gain 

0^ - 0^ « Gain 

HOj^ and i{o^ uould have to bo accepted and 1102 and U03 v.'ould have to be rejected • 

for a t to be slgnificaut at t;ie .05 level uith 33 degrees of freedom, it 
mucJt yield a value equal to or greater than 1,6J. 

Tae t on Hoj^ yielded a t *« .2?A and v;as; tiierefore, accoiited. H02 yielded 
a It « 2,636 and Ho 3 yielded a t^ « 2»41A vere rejected \i\ favor of tae alterna- 
tlves* Ho^ yielded a Jt a .081 and uas accepted/ 'fiU^ the findings support 
the theoretical hypothesis of the effects of the mediation. invest ie.atoro 

suspected that t!*ou|jU the mdla had been rated as priwary that the subjc^cts 
draun from the intermediate l6vel population performed significantly different 
than those dravm from the Prltnary population* Three additional Null Hypotliesejg 
t:ere formulated around the question of differences bett/ecn the Intermediate and 
Primary subjects. 

KOjj There v?lll bu no difference in performance on bet^/een Primary 

and Intermediate sub ject3» 
Ho^ Taerc v;lll be no differoi^ce in performance on 02 beti?een Primary 

and Intermediate subjects* 
lio^ There will be no difference in performance on O3 het\/een Primary 

and Intermediate suljects. 



Ho^ yielded a » 1.174 and Ho^, yi<jldcd a « At the ,05 level 

for two tailed test with df « 13, the t^ value uould have to equal or oxceod 
1.73» Ho^ was accepted and Ho^ rejected. Ho^ yielded a t^ « 2-571 and ^'as 
rejected. We may, therefore, logically conclude t'lat the rejection of II03 is 
a function of the difference in performance between the Primary and Intenaodiate 
subjects as measured by the observational instrument , Vue Intermediate 0^ 
yielded a « 4.9 and SD » 1,64 and the Intermediate O2 yielded a 3c - 5.1 and 
SD « >03. The A of the Prlrmry fja^ple remained at 4.1 while the SI* » 1.22 for 

and 1.37 for Examination of the mean and standard deviation for the tro 

groups siiOv; that the Intermediate group acquired nore correct re3,:?onse3 oa 03 
as a result of taking the firiJt pretest 0^ than the Primary group. 
Discussion 

Logically /v?e must conclude t^iat post test perforais.ice in this study vas a 
function of the viewing of the filmstrlp 'Ttie Ilephant. One must also conclude 
that there is an ago level or maturity level below rhlch cognitive gain 
traceable to a mediated presentation may not occur. The question atise;^ as to 
what effect, sans entertainment, Is to be derived from providing madiated 
material to Primary level educable mentally retarded student. 
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